1. Field of the Invention
The present invention relates to an image processing method and an image processor for estimating a signal value at all pixel positions by employing image data which does not have a signal value representing all colors at all pixel positions, like image data representing a color image obtained by an imaging device such as a single plate charge-coupled device (CCD), and to a computer readable storage medium storing a program for causing a computer to execute the image processing method.
2. Description of the Related Art
As an imaging device such as a CCD employed in a digital camera, an imaging device in which a plurality of kinds of photoelectric conversion elements differing in spectral sensitivity are alternately arrayed on the same surface (hereinafter referred to a single plate CCD) is known. In the case of the single plate CCD, in which photoelectric conversion elements having spectral sensitivities for red (R), green (G), and blue (B), i.e., photoelectric conversion elements for an R-channel, a G-channel, and a B-channel are alternately arrayed, a set of three continuous photoelectric conversion elements for an R-channel, a G-channel, and a B-channel constitutes a single pixel. In such a single plate CCD, however, shift of color alignment or a false color sometimes occurs because the R-signal, G-signal, and B-signal values at each pixel cannot be obtained at the same pixel position. Also, since the number of photoelectric conversion elements for channels is less than the total number of pixels constituting a single plate CCD, an image with high resolution cannot be obtained. For instance, in a single plate CCD where photoelectric conversion elements for an R-channel, a G-channel, and a B-channel are alternately arrayed, the number of photoelectric conversion elements for channels is only one-third the total number of elements and therefore the resolution becomes ⅓ compared with a monochrome imaging device having the same number of elements. Because of this, a method has been proposed in which a signal value is computed by an interpolating process at a position where photoelectric conversion elements for an R-channel, a G-channel, and a B-channel are not present. However, there are cases where only an interpolating process causes a false color to occur at a position where a signal value changes considerably. In this case, although the occurrence of a false color can be prevented by using an optical low-pass filter in an imaging system, or by performing a smoothing process on an image signal through a low-pass filter, there is a problem that the resolution will degrade.
Here, human visual sense characteristics are higher in sensitivity with respect to brightness than with respect to color. For this reason, a method has been proposed in which a high-frequency brightness signal (which represents the brightness of each pixel) and a low-frequency color signal (obtained by the above-mentioned interpolating process and the smoothing process which employs a low-pass filter) are generated from a color image signal obtained by a single plate CCD and color image signals are reconstituted by the generated brightness signal and color signal (Japanese Unexamined Patent Publication No. 10(1998)-200906). This method can obtain color image signals capable of reproducing an image whose resolution is apparently high, because much information about a brightness component whose sensitivity is high in the human visual sense characteristic can be given.
Incidentally, a CCD with a honeycomb array of pixels arrayed checkerwise as shown in FIG. 16, for example, is known as a single plate CCD (e.g., Japanese Unexamined Patent Publication No. 10(1998)-136391). This is also called a checkered pixel array. There is known another CCD that has a Bayer array of pixels arrayed in square form, as shown in FIG. 17. This is also called a square pixel array. The single plate CCD having such an array of pixels also has the problem of false color. Furthermore, a method is known which is based on the assumption that the ratio of an r-signal, a g-signal, and a b-signal is approximately constant at local regions of an image (Japanese Unexamined Patent Publication No. 9(1997)-214989). In order to remove a false color from a light-quantity based signal obtained by the aforementioned CCD having a Bayer array of pixels, in the vertical or horizontal direction of the CCD a g-signal is multiplied by the ratio of a r-signal and a g-signal obtained at the adjacent line, whereby an r-signal at that line is computed. More specifically, in an array of pixels shown in FIG. 18, in order to compute an r-signal r12 at a g12-pixel position, a g11-signal at an r11-pixel position is first computed by an equation of (g6+g16)/2. Based on an assumption of r11:g11=r12:g12, the r12-signal at the g12-pixel position is computed by an equation of r12=g12xc3x97r11/g11.
However, in the method disclosed in the above-mentioned Japanese Unexamined Patent Publication No. 10(1998)-200906), etc., even if a smoothing process through a low-pass filter is performed on an image signal obtained by the single plate CCD, a high-frequency component in the actual image has been folded back into the image. Because of this, moire due to aliasing noise cannot be removed, and consequently, a false color cannot be sufficiently removed.
On the other hand, the method disclosed in the aforementioned Japanese Unexamined Patent Publication No. 9(1997)-214989 is capable of removing a false color effectively. Particularly, this method is based on the assumption that the light-quantity ratio of r:g:b is constant at local regions of an image, and in the case of an analog signal in which the ratio of an R-signal, a G-signal, and a B-signal obtained is proportional to a quantity of light, a false signal in an image signal obtained by a CCD having a Bayer array of pixels can be effectively removed. However, an image signal obtained by a digital camera does not become r:g:b=R:G:B, because, when light quantities r, g, b are converted to a digital R-signal, G-signal, and B-signal, a signal value is represented by an exponential or logarithmic value such as R=r0.45 and R=log(r), in order to reduce quantum errors and input a signal to a video circuit in a computer system. Because of this, the method disclosed in the aforementioned Publication No. 9(1997)-214989 can remove a false color from an analog signal whose signal value is proportional to a quantity of light, but cannot remove a false color in the case where a signal value is represented by an exponential or logarithmic value of a quantity of light. In addition, a false color occurs not only in a single plate CCD having a Bayer array of pixels but also in a single plate CCD having a honeycomb array of pixels.
The present invention has been made in view of the drawbacks found in the aforementioned methods. Accordingly, it is an object of the invention to provide an image processing method and an image processor which are capable of reducing the occurrence of a false color even if a signal is of any type. Another object of the invention is to provide a computer readable storage medium in which a program for causing a computer to execute the image processing method is stored.
For example, in the case where each signal value, which constitutes image data obtained by an imaging device such as a single plate CCD, is represented by the exponential value or logarithmic value of a light quantity, the image processing method according to the present invention has been made based on the assumption that the differences between signal values become equal at local regions of an image.
In accordance with the present invention, there is provided a method of processing image data,
the data representing an image that includes a first pixel, a second pixel, and a third pixel that have a first signal value, a second signal value, and a third signal value that have different spectral distributions,
the first pixel and the second pixel being alternately arrayed in a predetermined direction to form a first line, the first pixel and the third pixel being alternately arrayed in the predetermined direction to form a second line, and the first line and the second line being alternately arrayed in a direction approximately orthogonal to the predetermined direction,
the method comprising a step of estimating at least one signal value among the first signal value, the second signal value, and the third signal value at all pixel positions, based on the first signal value, the second signal value, and the third signal value,
wherein the second signal value on the second line is estimated based on a difference between the first signal value and the second signal value on the first line adjacent to the second line.
That is, for instance, when it is assumed that a single plate CCD having a honeycomb array of pixels such as that shown in FIG. 16 has spectral sensitivities corresponding to R, G, and B, image data consisting of color signals (R, G, and B) is obtained. Here, the array of pixels of an image represented by this image data will be described with reference to FIG. 16. Assume that the first through the third pixels and the first through the third signal values correspond to G, R, and B, respectively. The first line is taken to be a GR-line of alternately arraying a G-pixel and an R-pixel from the upper left to the bottom right, while the second line is taken to be a GB line of alternately arraying a G-pixel and a B-pixel in the same direction as the first line. In this case, the image processing method according to the present invention is characterized in that an R-signal value on the GB line is estimated based on the difference between a G-signal and an R-signal on the GR line.
For the purposes of making the present invention more understandable, the first through the third pixels and the first through the third signal values correspond to G, R, and B, respectively. Also, the first line and the second line correspond to the GR line and the GB line, respectively. While, in the following description, reference numerals and characters employed in the embodiment of the present invention shown in FIG. 4 will be applied within parentheses with respect to G, B, and R, the present invention is not to be limited to this array of pixels of an image represented by image data.
In the aforementioned image processing method of the present invention, it is preferable that a second (R06) signal value at the first (G06) pixel on the second (GB) line be estimated based on a difference (R09xe2x88x92G09) between a second (R09) signal value at the second (R09) pixel adjacent to the first (G06) pixel on the first (GR) line, and the first (G09) signal value at the second (R09) pixel computed by performing a one-dimensional interpolation computation on the first (G05, G13) signal value on the first (GR) line.
Also, in this case it is preferable that a second (R06) signal value at the first (G06) pixel on the second (GB) line be estimated by adding the difference (R09xe2x88x92G09) to a first (G06) signal value at the first (G06) pixel on the second (GB) line.
Furthermore, in the image processing method of the present invention, it is preferable that a second (R10) signal value at the third (B10) pixel on the second (GB) line be estimated based on a difference (R13xe2x88x92G13) between a first (G13) signal value at the first (G13) pixel adjacent to the third (B13) pixel on the first (GR) line, and the second (R13) signal value at the first (G13) pixel computed by performing a one-dimensional interpolation computation on the second (R09, R17) signal value on the first line (GR).
In this case it is preferable that a first (G10) signal value at the third (B10) pixel on the second (GB) line be computed by performing a one-dimensional interpolation computation on the first (G06, G14) signal value on the second (GB) line, and it is also preferable that a second (R10) signal value at the third (B10) pixel on the second (GB) line be estimated by adding the difference (R13xe2x88x92G13) to the computed first (G10) signal value.
Furthermore, in the image processing method of the present invention, it is preferable that the difference be an average value of the differences (e.g., R09xe2x88x92G09, R03xe2x88x92G03, etc.) on the two first (GR) lines adjacent to the second (GB) line.
Moreover, in the image processing method of the present invention, it is preferable that the third (B) signal value on the first (GR) line be estimated based on a difference between the first (G) signal value and the third (B) signal value on the second (GB) line adjacent to the first (GR) line.
Also, in this case it is preferable that a third (B13) signal value at the first (G13) pixel on the first (GR) line be estimated based on a difference (B10xe2x88x92G10) between a third (B10) signal value at the third (B10) pixel adjacent to the first (G13) pixel on the second (GB) line, and the first (G10) signal value at the third (B10) pixel computed by performing a one-dimensional interpolation computation on the first (G06, G14) signal value on the second (GB) line.
Furthermore, in this case it is preferable that a third (B13) signal value at the first (G13) pixel on the first (GR) line be estimated by adding the difference (B10xe2x88x92G10) to a first (G13) signal value at the first (G13) pixel on the first (GR) line.
In addition, in the image processing method of the present invention, it is preferable that a third (B13) signal value at the second (R09) pixel on the first (GR) line be estimated based on a difference (B06xe2x88x92G06) between a first (G06) signal value at the first (G06) pixel adjacent to the second (R09) pixel on the second (GB) line, and the third (B06) signal value at the first (G06) pixel computed by performing a one-dimensional interpolation computation on the third (B02, B10) signal value on the second (GB) line.
In this case it is preferable that a first (G09) signal value at the second (R09) pixel on the first (GR) line be computed by performing a one-dimensional interpolation computation on the first (G05, G13) signal value on the first (GR) line, and it is also preferable that a third (B09) signal value at the second (R09) pixel on the first (GR) line be estimated by adding the difference (B06xe2x88x92G06) to the computed first (G09) signal value.
In addition, it is preferable that the difference be an average value of the differences (e.g., B10xe2x88x92G10, B16xe2x88x92G16, etc.) on the two second (GB) lines adjacent to the first (GR) line.
Furthermore, in the image processing method of the present invention, it is preferable that
when, in a direction (direction of arrow B) orthogonal to the predetermined direction (direction of arrow A), the first (G) and second (B) pixels are alternately arrayed and the first (G) and third (B) pixels are alternately arrayed so that the first (GR) line and the second (GB) line are formed in the orthogonal direction,
at least one signal value among the first signal value, the second signal value, and the third signal value be estimated by switching the first and second lines in the predetermined direction and the first and second lines in the orthogonal direction in accordance with a pixel position for estimating a 10, signal value.
In this case it is preferable that the aforementioned switching be performed based on a scale value which represents a direction of a change in a signal value at the pixel position for estimating a signal value.
The words xe2x80x9cscale valuexe2x80x9d refer to the direction of a change in a signal value at a pixel position for estimating a signal value. For example, the scale value can be represented by the quantity of a change in a signal value at a pixel position adjacent to a pixel position for estimating a signal value, in the aforementioned predetermined direction and the aforementioned orthogonal direction.
Furthermore, in the image processing method of the present invention, it is preferable that
when, in a direction (direction of arrow B) orthogonal to the predetermined direction (direction of arrow A), the first (G) and second (B) pixels are alternately arrayed and the first (G) and third (B) pixels are alternately arrayed so that the first (GR) line and the second (GB) line are formed in the orthogonal direction,
a predetermined-direction estimated value, based on the first and second lines in the predetermined direction, be computed and an orthogonal-direction estimated value based on the first and second lines in the orthogonal direction is computed. It is also preferable that at least one signal value among the first signal value, the second signal value, and the third signal value be estimated, by weighting and adding the predetermined-direction estimated value and the orthogonal-direction estimated value by a predetermined weighting coefficient in accordance with a pixel position for estimating a signal value.
In this case it is preferable that the predetermined weighing coefficient be computed based on a scale value which represents a direction of a change in a signal value at the pixel position for estimating a signal value.
Here, the xe2x80x9cpredetermined-direction estimated valuexe2x80x9d and the xe2x80x9corthogonal-direction estimated valuexe2x80x9d refer to at least one signal value among the first through the third signal values estimated in the predetermined direction and the orthogonal direction.
In addition, in the image processing method of the present invention, it is preferable that an array of pixels on the first (GR) line relatively shift out of position by approximately one-half a pixel in the predetermined direction with respect to an array of pixels on the second (GB) line so that the first, second, and third pixels are arrayed checkerwise.
In the case of such an array of pixels, it is preferable that signal values at all pixel positions be estimated and that signal values at vacant pixel positions be estimated based on the estimated signal values.
Here, the words xe2x80x9cvacant pixel positionxe2x80x9d mean a position that can be regarded as a position having no signal value, such as a pixel position between R and B or between G and G in the horizontal direction of FIG. 16, when a honeycomb array of pixels in a single plate CCD, such as the one shown in FIG. 16, is taken to be a square array of pixels.
In the image processing method according to the present invention, it is preferable that the first signal value, the second signal value, and the third signal value be any color signal of green, blue, and red, respectively. It is also preferable that the first signal value, the second signal value, and the third signal value be any color signal of yellow, green, and cyan, respectively.
Moreover, in the image processing method of the present invention, the aforementioned image data may be obtained by an imaging device having an imaging surface. In this case the imaging surface is formed by arraying a first photoelectric conversion element, a second photoelectric conversion element, and a third photoelectric conversion element having different spectral sensitivities, on a single surface. The first and second photoelectric conversion elements are alternately arrayed in a predetermined direction to form a first line, and the first and third photoelectric conversion elements are alternately arrayed in the predetermined direction to form a second line. The first line and the second line are alternately arrayed a direction approximately orthogonal to the predetermined direction.
In accordance with the present invention, there is provided a second method of processing image data,
the data representing an image that includes a first pixel, a second pixel, and a third pixel that have a first signal value, a second signal value, and a third signal value that have different spectral distributions,
the first pixel and the second pixel being alternately arrayed in a predetermined direction to form a first line, the first pixel and the third pixel being alternately arrayed in the predetermined direction to form a second line, and the first line and the second line being alternately arrayed in a direction approximately orthogonal to the predetermined direction,
the method comprising the steps of:
obtaining estimated data by estimating at least one signal value among the first signal value, the second signal value, and the third signal value at all pixel positions on the image, by the method as set forth in any one of claims 1 through 36;
generating a high-frequency brightness signal which represents brightness information of a high frequency of the image data;
converting the estimated image data to an estimated brightness signal and an estimated color difference signal which represent brightness information and color difference information of the estimated image data;
obtaining an added brightness signal by adding the estimated brightness signal and the high-frequency brightness signal; and
employing the added brightness signal and the estimated color difference signal as a brightness color-difference signal of the image data.
Here, the method of generating a high-frequency brightness signal from image data can adopt a method of employing the first through the third signal values constituting image data, as they are, as brightness signals and performing a filtering process on the brightness signals through a high-pass filter, or methods disclosed, for example, in Japanese Unexamined Patent Publication Nos. 5(1993)-228108, 6(1994)-30444, 6(1994)-225343, and 8(1996)-9199.
In the aforementioned second method according to the present invention, it is preferable that the high-frequency brightness signal be generated by performing a filtering process on the image data through a high-pass filter for cutting a frequency component of a frequency band of the estimated brightness signal.
Also, in the aforementioned second method according to the present invention, it is preferable that the high-frequency brightness signal be generated by performing a filtering process on the image data through a band-pass filter having a predetermined passband characteristic.
Here, the words xe2x80x9cpredetermined passband characteristicxe2x80x9d mean, for example, a characteristic of cutting a frequency component of a frequency band corresponding to the noise component of image data.
In accordance with the present invention, there is provided an image processor for processing image data,
the data representing an image that includes a first pixel, a second pixel, and a third pixel that have a first signal value, a second signal value, and a third signal value that have different spectral distributions,
the first pixel and the second pixel being alternately arrayed in a predetermined direction to form a first line, the first pixel and the third pixel being alternately arrayed in the predetermined direction to form a second line, and the first line and the second line being alternately arrayed in a direction approximately orthogonal to the predetermined direction,
the image processor comprising estimation means for estimating at least one signal value among the first signal value, the second signal value, and the third signal value at all pixel positions, based on the first signal value, the second signal value, and the third signal value,
wherein the estimation means estimates the second signal value on the second line, based on a difference between the first signal value and the second signal value on the first line adjacent to the second line.
In the image processor according to the present invention, it is preferable that the estimation means estimate a second signal value at the first pixel on the second line, based on a difference between a second signal value at the second pixel adjacent to the first pixel on the first line, and the first signal value at the second pixel computed by performing a one-dimensional interpolation computation on the first signal value on the first line.
In this case it is preferable that the estimation means estimate a second signal value at the first pixel on the second line, by adding the difference to a first signal value at the first pixel on the second line.
In addition, in the image processor according to the present invention it is preferable that the estimation means estimate a second signal value at the third pixel on the second line, based on a difference between a first signal value at the first pixel adjacent to the third pixel on the first line, and the second signal value at the first pixel computed by performing a one-dimensional interpolation computation on the second signal value on the first line.
In this case the estimation means may compute a first signal value at the third pixel position on the second line, by performing a one-dimensional interpolation computation on the first signal value on the second line, and may also estimate a second signal value at the third pixel position by adding the difference to the computed first signal value.
Also, the difference may be an average value of the differences on the two first lines adjacent to the second line.
In addition, in the image processor according to the present invention, it is preferable that the estimation means estimate the third signal value on the first line, based on a difference between the first signal value and the third signal value on the second line adjacent to the first line.
In this case it is preferable that the estimation means estimate a third signal value at the first pixel on the first line, based on a difference between a third signal value at the third pixel adjacent to the first pixel on the second line, and the first signal value at the third pixel computed by performing a one-dimensional interpolation computation on the first signal value on the second line.
Also, in this case it is preferable that the estimation means estimate a third signal value at the first pixel on the first line, by adding the difference to a first signal value at the first pixel on the first line.
In addition, in the image processor according to the present invention, it is preferable that the estimation means estimate a third signal value at the second pixel on the first line, based on a difference between a first signal value at the first pixel adjacent to the second pixel on the second line, and the third signal value at the first pixel computed by performing a one-dimensional interpolation computation on the third signal value on the second line.
In this case the estimation means may compute a first signal value at the second pixel position on the first line, by performing a one-dimensional interpolation computation on the first signal value on the first line, and may also estimate a third signal value at the second pixel position, by adding the difference to the computed first signal value.
Also, it is preferable that the estimation means compute the difference as an average value of the differences on the two second lines adjacent to the first line.
Furthermore, in the image processor according to the present invention, it is preferable that
when, in a direction orthogonal to the predetermined direction, the first and second pixels are alternately arrayed and the first and third pixels are alternately arrayed so that the first line and the second line are formed in the orthogonal direction,
the estimation means estimates at least one signal value among the first signal value, the second signal value, and the third signal value, by switching the first and second lines in the predetermined direction and the first and second lines in the orthogonal direction in accordance with a pixel position for estimating a signal value.
In this case it is preferable that the aforementioned switching be performed based on a scale value which represents a direction of a change in a signal value at the pixel position for estimating a signal value.
Furthermore, in the image processor according to the present invention, it is preferable that
when, in a direction orthogonal to the predetermined direction, the first and second pixels are alternately arrayed and the first and third pixels are alternately arrayed so that the first line and the second line are formed in the orthogonal direction,
the estimation means comprise
means for computing a predetermined-direction estimated value based on the first and second lines in the predetermined direction and an orthogonal-direction estimated value based on the first and second lines in the orthogonal direction, and
means for estimating at least one signal value among the first signal value, the second signal value, and the third signal value, by weighting and adding the predetermined-direction estimated value and the orthogonal-direction estimated value by a predetermined weighting coefficient in accordance with a pixel position for estimating a signal value.
In this case it is preferable that the predetermined weighing coefficient be computed based on a scale value which represents a direction of a change in a signal value at the pixel position for estimating a signal value.
In addition, in the image processor according to the present invention, it is preferable that an array of pixels on the first line relatively shift out of position by approximately one-half a pixel in the predetermined direction with respect to an array of pixels on the second line so that the first, second, and third pixels are arrayed checkerwise.
In this case it is preferable that the estimation means estimate signal values at all pixel positions and also estimates signal values at vacant pixel positions, based on the estimated signal values.
In the image processor according to the present invention, it is preferable that the first signal value, the second signal value, and the third signal value be any color signal of green, blue, and red, respectively. It is also preferable that the first signal value, the second signal value, and the third signal value be any color signal of yellow, green, and cyan, respectively.
Also, in the image processor according to the present invention, it is preferable that the image data be obtained by an imaging device having an imaging surface. In this case the imaging surface is formed by arraying a first photoelectric conversion element, a second photoelectric conversion element, and a third photoelectric conversion element having different spectral sensitivities, on a single surface. The first and second photoelectric conversion elements are alternately arrayed in a predetermined direction to form a first line, and the first and third photoelectric conversion elements are alternately arrayed in the predetermined direction to form a second line. Furthermore, the first line and the second line are alternately arrayed in a direction approximately orthogonal to the predetermined direction.
In accordance with the present invention, there is a second image processor for processing image data,
the data representing an image that includes a first pixel, a second pixel, and a third pixel that have a first signal value, a second signal value, and a third signal value that have different spectral distributions,
the first pixel and the second pixel being alternately arrayed in a predetermined direction to form a first line, the first pixel and the third pixel being alternately arrayed in the predetermined direction to form a second line, and the first line and the second line being alternately arrayed in a direction approximately orthogonal to the predetermined direction,
the image processor comprising:
interpolation means for obtaining estimated image data by estimating at least one signal value among the first signal value, the second signal value, and the third signal value at all pixel positions on the image, by the image processor as set forth in any one of claims 40 through 71;
high-frequency brightness signal generation means for generating a high-frequency brightness signal which represents brightness information of a high frequency of the image data;
brightness color-difference conversion means for converting the estimated image data to an estimated brightness signal and an estimated color difference signal which represent brightness information and color difference information of the estimated image data; and
addition means for obtaining an added brightness signal by adding the estimated brightness signal and the high-frequency brightness signal;
wherein the added brightness signal and the estimated color difference signal are employed as a brightness color-difference signal of the image data.
In the second image processor according to the present invention, it is preferable that the high-frequency brightness signal generation means generate the high-frequency brightness signal by performing a filtering process on the image data through a high-pass filter for cutting a frequency component of a frequency band of the estimated brightness signal.
In addition, in the second image processor according to the present invention, it is preferable that the high-frequency brightness signal generation means generate the high-frequency brightness signal by performing a filtering process on the image data through a band-pass filter having a predetermined passband characteristic.
Note that a program for causing a computer to execute the image processing method and image processor of the present invention may be stored in a computer readable storage medium. Also, the image processor of the present invention may be provided in an output device such as a printer, etc.
According to the image processing method and the image processor of the present invention, when a signal value constituting image data is represented by an exponential value or logarithmic value with respect to a quantity of light, a signal value at each pixel position is computed based on the assumption that the difference between RGB-signals is constant at local regions of an image represented by image data. Therefore, even if a signal value constituting image data is represented by an exponential value or logarithmic value, signal values at all pixel positions can be computed without the occurrence of a false color. As a result, the present invention is capable of obtaining an image with no false color and with high resolution.
In addition, a signal value at each pixel position is computed by weighting and adding signal values computed in a predetermined direction and a direction orthogonal to the predetermined direction, in accordance with a direction in which a signal value changes. As a result, the occurrence of a false color can be prevented regardless of the direction of a change in a signal value, and the occurrence of an artifact can also be prevented.
Furthermore, when pixels are arrayed checkerwise, signal values at pixels arrayed in square form can be computed by computing signal values at vacant pixel positions.
Moreover, in accordance with a second image processing method and image processor of the present invention, estimated image data is obtained, by estimating at least one signal value among the first signal value, the second signal value, and the third signal value at all pixel positions of an image represented by image data by the first image processing method and image processor of the present invention. A high-frequency brightness signal representing the brightness information of a high frequency of image data is generated and the estimated image data is converted to an estimated brightness signal and an estimated color difference signal, which represent the brightness information and color difference information of the estimated image data. Also, an added brightness signal is obtained by adding the estimated brightness signal and the high-frequency brightness signal. Here, since the occurrence of a false color in the estimated image data has been reduced, the image of a scene from which image data is acquired can be reproduced without the occurrence of a false color. On the other hand, while the high frequency component of image data represents with fidelity the high frequency component of a scene from which image data is acquired, a false color is contained in the low frequency component. Therefore, the high-frequency brightness signal which represents the brightness information of the high frequency of image data becomes a signal that represents the high frequency component of a scene with fidelity. For this reason, the added brightness signal, obtained by adding the high-frequency brightness signal and the estimated brightness signal, represents the high frequency component of a scene with fidelity, and for the low frequency component, the occurrence of a false color has been reduced. As a result, an image with higher resolution can be obtained by reproducing the image, based on the added brightness signal and the estimated color difference signal.
Assuming the ratio of the first signal value, the second signal value, and the third signal value does not change sharply independently of positions on an image, the second image processing method and image processor of the present invention is capable of obtaining an image having even higher resolution. However, when a change in the above-mentioned ratio is great, a false signal is contained in the high-frequency brightness signal. Therefore, if the added brightness signal is generated from this high-frequency brightness signal and if an image is reproduced by the added brightness signal and the estimated color difference signal, gray noise will appear in the image. Therefore, in such a case, if the high-frequency brightness signal is generated by the use of a band-pass filter for cutting the frequency component of a frequency band corresponding to the gray noise component, gray noise can be reduced and image resolution can also be enhanced.